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DETAILED ACTION 
Response to Amendment 

1 . This Office action is responsive to the Request for Continued Examination (RCE) 
filed under 37 CFR §1 .53(d) on 8/16/07. Applicants have properly set forth the RCE, 
which has been entered into the application, and an examination on the merits follows 
herewith. The Examiner acknowledges Applicant's amendments to claims 1, 2, 6, 7, 9- 
11, 29 and 56 and the cancellation of claim 38. All pending claims have been rejected 
as being obvious over prior arts disclosed in the field of video summarization, where the 
previous rejection has been maintained. 

Claim Rejections - 35 USC §112 
The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

2. Claims 1, 6, 7, 9 and 10 are rejected under 35 U.S.C. 112, first paragraph, as 
failing to comply with the written description requirement. The claim(s) contains subject 
matter which was not described in the specification in such a way as to reasonably 
convey to one skilled in the relevant art that the inventor(s), at the time the application 
was filed, had possession of the claimed invention. The claims disclose that the 
plurality of segments is bounded by a spatial region on the first portion of the display. 
The specification does not clearly convey the description of the bounded spatial region 
and does not describe the association between the first portion, second portion and the 
bounded spatial region. 
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3. Claim 1 1 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. The body of the disclosure has not clearly 
described the user moveable scroll bar and the features associated with this scroll bar. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-7, 9-34, 36, 37, 39-56 and 58-86 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over "Adjustable Filmstrips and Skims as Abstractions for a 
Digital Video Library" (Christel et al.), herein referred to as Christel, "Bayesian Modeling 
Of Video Editing And Structure: Semantic Features For Video Summarization And 
Browsing" (Vasconcelos et al.), herein referred to as Vasconcelos and U. S. Patent No. 
6, 880, 171 B1 (Ahmad et al.), herein referred to as Ahmad. 

Referring to claim 1 , Christel discloses a method of presenting information 
regarding a video with a plurality of frames (page 2, column 2, lines 1 1-19). Christel 
discloses summarizing a video by identifying a plurality of segments of the video based 
upon an event (page 4, column 2, lines 28-32). Christel discloses that each of the 
segments includes a plurality of sequential frames of the video (page 4, column 2, lines 
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8-20). Christel discloses displaying a summarization in a first portion of a display 
(Figure 5), wherein the first portion discloses a snapshot of a skim and indicating the 
sequences that make up the skim at the bottom of the window. Christel discloses 
displaying a graphical user interface on a second portion of the display, sequentially 
indicating the relative location of the plurality of segments within the summarization 
relative to at least one other of the segments as each of the plurality of segments is 
displayed (Figure 5), wherein the second portion displays the match locations in relation 
to the location of plurality of segments within the summarization. The plurality of 
segments is bounded blocks of spatial region that are displayed on the first display 
portion of the display. Christel discloses displaying to the user the relative location for a 
first type of content of the video using a first visual indication (Figure 5), wherein this is 
represented as the gray blocks at the bottom of the window and displaying the relative 
location for second type of content of the video using a second visual indication different 
from the first visual indication (Figure 5), wherein the white lines showing the match 
locations represents the second visual indication. Christel discloses receiving from the 
user through interaction with the graphical user interface a selection of one of the 
plurality of segments (Figure 6 and page 5, column 2, lines 1-7). The user interaction 
allows for viewing desired segment based on the initial summary of the segment that 
the user has viewed. In response to the selection, presenting a selected one of the 
plurality of segments and not presenting at least one other of the plurality of segments. 
Once the user has determined that a certain segment is to be viewed, user interaction 
with the display of Figure 6 allows for that distinct segment to be viewed while not 
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viewing other segments, which can be skimmed through. Christel does not explicitly 
disclose that the event being captured is characterized by a play, wherein this play is of 
a sport. It would have been obvious for one skilled in the art at the time of the invention 
that the event being captured is characterized by a play within a sport. Christel includes 
all types of video when discussing capturing of important events from a video, wherein 
based on the content of the video, any important events relevant to that specific video 
would be captured. Christel discloses that the goal of the invention is to display the 
essential content of a video, and wherein it is obvious that when the video is that of a 
sporting event, the essential content would be the plays and hence it would be obvious 
for Christel to captures these plays, wherein these plays are those that are part of a 
sporting event. The plays clearly represent characterizations of semantic events of the 
sporting event, representing an event that has clear association with the context of the 
sporting event and describing specific events that have occurred. Christel does not 
disclose two different characterizations of semantic events, which are indicated by two 
different visual means. Vasconcelos discloses identifying specific semantic events and 
displaying the identified data through visual indications in a timeline (Figure 2). It would 
have been obvious to one skilled in the art at the time of the invention to learn from 
Vasconcelos to identify semantic events and display the identified data in a timeline. 
With Vasconcelos teaching that identification of semantic data will allow users to quickly 
access relevant and desired portions of a large video (page 5, column 1, lines 10-12) 
provides a motivation for one skilled in the art to learn from Vasconcelos. Therefore, 
based on the information provided by Vasconcelos, Christel's video system would have 
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been motivated to learn from Vasconcelos to identify semantic events from within video 
data. One skilled in the art at the time of the invention would have been motivated to 
learn from Vasconcelos to identify semantic events from video data and display the data 
in a timeline format. 

Christel and Vasconcelos do not disclose displaying the identified semantic 
events in the timeline with different visual indications for two different characterizations 
of semantic event. Ahmad discloses obtaining summary data related to video and audio 
data, with visual means for indicating a relative location for a first characterization of 
data in a video data and displaying the relative location for a second characterization of 
data in the video using a second visual indication different from the first visual indication 
(column 16, lines 54-65). It would have been obvious for one skilled in the art, at the 
time of the invention to learn from Ahmad to teach two different visual indications for two 
different characterizations or descriptions of information. Christel, Vasconcelos and 
Ahmad disclose systems for summarizing and obtaining relevant data from video and 
audio data, further displaying timelines for indicating relevant data. Ahmad further 
displays detailed bar information to indicate different visual means for different 
characterizations of data. It would have been obvious for one skilled in the art, at the 
time of the invention to learn from Ahmad to implement detailed bar data to further 
provide more detailed information concerning the video and summary data obtained. 
Vasconcelos discloses an objective for providing semantic information so as to display 
data without overwhelming the user. Vasconcelos' timeline includes the identification of 
different characterizations of semantic events through letters A, D, C and S would 
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further be efficiently handled by providing two different visual indications for different 
characterizations of semantic events. The display means of Ahmad would allow for 
easier accessing of desired data with clear displays mean for quickly identifying specific 
data. Therefore, one skilled in the art would have been motivated to learn from Ahmad 
to use different visual indications for displaying different characterizations of semantic 
events. 

Referring to claim 2, Christel, Vasconcelos and Ahmad disclose that the first and 
second semantic characterizations of the play temporally overlap in the summarization 
(Christel, Figures 5 and 6), where the summarization sections discloses two types of 
data with each of them temporally overlapping at distinct points. 

Referring to claim 3, Christel discloses a generally rectangular region where each 
of the plurality of segments is indicated within the generally rectangular region (Figure 
5). 

Referring to claim 4, Christel discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 5). 

Referring to claim 5, Christel discloses that the size of the regions between each 
of the plurality of segments is indicated in a manner such that the regions between with 
a greater number of frames are larger than plurality of segments with a lesser number of 
frames (Figure 5). 
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Referring to claim 6, Christel discloses that the user selects one of the plurality of 
segments by interacting with the graphical user interface at point within the displayed 
bounded spatial region associated with the selected one of the plurality of segments 
(Figure 6). The user selects one of the desired segments through the user interface of 
Figure 6, the user selector selected at a distinct point selects the desired one of the 
plurality of segments. 

Referring to claim 7, Christel discloses presentation of a selected one of the 
plurality of segments begins at a first frame of the segment irrespective of which point 
within the displayed bounded spatial region that the user interacted with (Figure 6 and 
page 5, column 2, lines 1-7). 

Referring to claim 9, Christel discloses that the presentation of a selected one of 
the plurality of segments begins at a frame of the segment temporally corresponding to 
the point within the displayed spatial region that the user interacted with (Figure 6). The 
segment selected begins at a first frame of the selected segment with all segments 
being temporally placed in relation to the bound spatial region. 

Referring to claim 10, Christel discloses including a selector by which the user 
may alternatively select a chosen of the presentation of a selected one of the plurality of 
segments beginning at the first frame of the segment irrespective of which point with the 
displayed bounded spatial region that the user interacted with (Figure 6 and page 5, 
column 2, lines 1-7). Christel also discloses presentation of a selected one of the 
plurality of segments beginning at a frame of the segment temporally corresponding to 
the point within the displayed bounded spatial region that the user interacted that the 
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user interacted with (Figure 6). The segment selected begins at a first frame of the 
selected segment with all segments being temporally placed in relation to the bound 
spatial region. 

Referring to claim 11, Christel discloses including a user moveable scroll bar on 
the graphical user interface indicating the relative temporal location of the currently 
presented frames of the summary (Figures 5 and 6). Christel discloses that the user 
selects one of the plurality of segments by moving the scroll bar over the selected one 
of the plurality of segments with the scroll bar snapping to the beginning of the selected 
one of the plurality of segments (Figures 5 and 6). 

Referring to claim 12, Christel discloses that at least two of the plurality of 
segments is temporally overlapping (page 4, column 2, lines 26-29). 

Referring to claim 13, Christel discloses that the temporally overlapping 
segments are visually indicated in a manner such that each of the overlapping 
segments is independently identifiable (page 4, column 2, lines 26-31). 

Referring to claim 14, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the pluralities of segments (Figure 6). 

Referring to claim 1 5, Christel discloses that one of the pluralities of segments is 
the segment most temporally adjacent to the portion of the video (Figure 6). 

Referring to claim 16, Christel discloses that one of the pluralities of segments is 
the next temporally related segment (Figure 6). 
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Referring to claim 17, Christel discloses that the plurality of segments is the 
previous temporally related segment (Figure 6). 

Referring to claim 18, Christel discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (page 5, column 1 , lines 5-1 1 ). 

Referring to claim 19, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (page 5, column 1 , 
lines 5-11 and Figure 6). 

Referring to claim 20, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 21, Christel discloses that a user selects a portion of video not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
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video not included within the plurality of segments, and wherein the user selects a 
portion of the video included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the video within the plurality 
of segments (page 5, column 1 , lines 5-1 1 and Figure 6). 

Referring to claim 22, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 23, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
maintain the differentiation of the overlapping time periods (Figures and page 4, column 
2, lines 26-31). 

Referring to claim 24, Christel discloses that the temporal information is 
displayed within a time line, wherein the temporal information is presented in a plurality 
of layers in a direction perpendicular to the length of the time line (Figure 5). 

Referring to claim 25, Christel discloses that the temporal information is 
displayed within a time line, wherein textual information is included within the time line 
(Figure 6). 

Referring to claim 26, Christel discloses that additional textual information is 
displayed upon selecting a portion of the time line (Figure 6). 

Referring to claim 27, Christel discloses that temporal information is displayed 
together with a time line, wherein additional textual information is displayed together 
with selecting a portion of the time line (Figure 6). 
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Referring to claim 28, Christel discloses that temporal information is displayed 
within a time line, wherein additional audio annotation is presented upon presenting a 
portion of the time line (page 4, column 1, lines 33-37 and Figure 6). 

Referring to claim 29, Christel discloses a method of presenting information 
regarding a video with a plurality of frames (page 2, column 2, lines 1 1-19). Christel 
discloses identifying a plurality of different segments of the video, where each of the 
segments includes a plurality of frames of the video (page 4, column 2, lines 8-32). 
Christel discloses displaying, simultaneously with a segment of the video, a graphical 
user interface including information regarding the temporal location of the segments, 
relative to at least one other of the segments of the video (Figure 5). Christel discloses 
displaying in an interactive display a temporal location for a first characterization of the 
content of the video using a first visual indication (Figure 5), wherein this is represented 
as the gray blocks at the bottom of the window and displaying the temporal location for 
second characterization of the content of the video using a second visual indication 
different from the first visual indication (Figure 5), wherein the white lines showing the 
match locations represents the second visual indication. Christel discloses displaying to 
a user a selector by which the user may interact with the interactive display to select for 
viewing selective identified ones of the plurality of segments (Figure 6 and page 5, 
column 2, lines 1-7). The selector of the user interface of Figure 6 allows for the user to 
select for viewing one of the desired segments of the plurality of segments. Christel 
also discloses receiving user-selections of identified ones of the plurality of segments 
(page 5, column 2, lines 1-7) and presenting these segments (Figure 6). Christel does 
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not disclose two different characterizations of semantic events through indicated by two 
different visual means. Vasconcelos discloses identifying specific semantic events and 
displaying the identified data through visual indications in a timeline (Figure 2). It would 
have been obvious to one skilled in the art at the time of the invention to learn from 
Vasconcelos to identify semantic events and display the identified data in a timeline. 
With Vasconcelos teaching that identification of semantic data will allow users to quickly 
access relevant and desired portions of a large video (page 5, column 1, lines 10-12) 
provides a motivation for one skilled in the art to learn from Vasconcelos. Therefore, 
based on the information provided by Vasconcelos, Christel's video system would have 
been motivated to learn from Vasconcelos to identify semantic events from within video 
data. One skilled in the art at the time of the invention would have been motivated to 
learn from Vasconcelos to identify semantic events from video data and display the data 
in a timeline format. 

Christel and Vasconcelos do not disclose displaying the identified semantic 
events in the timeline with different visual indications for two different types of semantic 
event. Ahmad discloses obtaining summary data related to video and audio data, with 
visual means for indicating a relative location for a first type of data in a video data and 
displaying the relative location for a second type of data in the video using a second 
visual indication different from the first visual indication (column 16, lines 54-65). It 
would have been obvious for one skilled in the art, at the time of the invention to learn 
from Ahmad to teach two different visual indications for two different types of 
information. Christel, Vasconcelos and Ahmad disclose systems for summarizing and 
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obtaining relevant data from video and audio data, further displaying timelines for 
indicating relevant data. Ahmad further displays detailed bar information to indicate 
different visual means for different types of data. It would have been obvious for one 
skilled in the art, at the time of the invention to learn from Ahmad to implement detailed 
bar data to further provide more detailed information concerning the video and summary 
data obtained. Vasconcelos discloses an objective for providing semantic information 
so as to display data without overwhelming the user. Vasconcelos' timeline includes the 
identification of different types of semantic events through letters A, D, C and S would 
further be efficiently handled by providing two different visual indications for different 
types of semantic events. The display means of Ahmad would allow for easier 
accessing of desired data with clear displays mean for quickly identifying specific data. 
Therefore, one skilled in the art would have been motivated to learn from Ahmad to use 
different visual indications for displaying different types of semantic events. 

Referring to claim 30, Christel discloses a generally rectangular region where 
each of the plurality of segments is indicated within the generally rectangular region 
(Figure 5). 

Referring to claim 31, Christel discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 5). 

Referring to claim 32, Christel discloses that the size of the regions between 
each of the plurality of segments is indicated in a manner such that the regions between 



Application/Control Number: 10/016,941 Page 15 

Art Unit: 2173 

with a greater number of frames are larger than plurality of segments with a lesser 
number of frames (Figure 5). 

Referring to claim 33, Christel discloses an indicator that indicates the current 
position within the temporal information of a currently displayed portion of the video 
(Figure 5). 

Referring to claim 34, Christel discloses the indicator changes location relative to 
the temporal information as the portion of the currently displayed portion of the video 
changes (Figure 5). 

Referring to claim 36, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said video 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the video (Figures 5 and 6). 

Referring to claim 37, Christel discloses that the indicator is modified to a portion 
of the video that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 38, Christel discloses the first type of content and second type 
of content are selectable by a user for presentation to the user (Figure 5, page 5, 
column 1, lines 9-12 and Figure 6). 

Referring to claims 39 and 60, Christel discloses that at least two of the plurality 
of segments is temporally overlapping (page 4, column 2, lines 26-29). 

Referring to claims 40 and 61, Christel discloses that the temporally overlapping 
segments are visually indicated in a manner such that each of the overlapping 
segments is independently identifiable (page 4, column 2, lines 26-31). 
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Referring to claim 41, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the pluralities of segments (Figure 6). 

Referring to claim 42, Christel discloses that one of the pluralities of segments is 
the segment most temporally adjacent to the portion of the video (Figure 6). 

Referring to claim 43, Christel discloses that one of the pluralities of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 44, Christel discloses that the plurality of segments is the 
previous temporally related segment (Figure 6). 

Referring to claim 45, Christel discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (page 5, column 1 , lines 5-1 1 ). 

Referring to claim 46, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (page 5, column 1 , 
lines 5-11 and Figure 6). 

Referring to claim 47, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
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presents the portion of the video within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 48, Christel discloses that a user selects a portion of video not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
video not included within the plurality of segments, and wherein the user selects a 
portion of the video included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the video within the plurality of segments 
(page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 49, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 50, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
maintain the differentiation of the overlapping time periods (Figure5 and page 4, column 
2, lines 26-31). 

Referring to claims 51 and 72, Christel discloses that the temporal information is 
displayed within a time line, wherein the temporal information is presented in a plurality 
of layers in a direction perpendicular to the length of the time line (Figure 5). 
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Referring to claims 52 and 73, Christel discloses that the temporal information is 
displayed within a time line, wherein textual information is included within the time line 
(Figure 6). 

Referring to claims 53 and 74, Christel discloses that additional textual 
information is displayed upon selecting a portion of the time line (Figure 6). 

Referring to claims 54 and 75, Christel discloses that temporal information is 
displayed together with a time line, wherein additional textual information is displayed 
together with selecting a portion of the time line (Figure 6). 

Referring to claims 55 and 76, Christel discloses that temporal information is 
displayed within a time line, wherein additional audio annotation is presented upon 
presenting a portion of the time line (page 4, column 1, lines 33-37 and Figure 6). 

Referring to claim 56, Christel discloses a method of presenting information 
regarding audio (page 2, column 2, lines 11-19). Christel discloses identifying a plurality 
of different segments of the audio, where each of the segments includes a temporal 
duration of the audio (page 4, column 2, lines 8-32). Christel discloses displaying, 
simultaneously with the segment of the audio, a graphical user interface including 
information regarding the temporal location of the segments, relative to at least one 
other of segment of the audio (Figure 5). Christel discloses displaying in an interactive 
display the temporal location for a first characterization of content of the audio using a 
first visual indication (Figure 5), wherein this is represented as the gray blocks at the 
bottom of the window and displaying the temporal location for second characterization 
of content of the audio using a second visual indication different from the first visual 
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indication (Figure 5), wherein the white lines showing the match locations represents 
the second visual indication. Christel discloses displaying to a user a selector by which 
the user may interact with the interactive display to select for viewing selective identified 
ones of the plurality of segments (Figure 6 and page 5, column 2, lines 1-7). The 
selector of the user interface of Figure 6 allows for the user to select for viewing one of 
the desired segments of the plurality of segments. Christel also discloses receiving 
user-selections of identified ones of the plurality of segments (page 5, column 2, lines 1- 
7) and presenting these segments (Figure 6). Christel does not disclose two different 
characterizations of semantic events through indicated by two different visual means. . 
Vasconcelos discloses identifying specific semantic events and displaying the identified 
data through visual indications in a timeline (Figure 2). It would have been obvious to 
one skilled in the art at the time of the invention to learn from Vasconcelos to identify 
semantic events and display the identified data in a timeline. With Vasconcelos 
teaching that identification of semantic data will allow users to quickly access relevant 
and desired portions of a large video (page 5, column 1, lines 10-12) provides a 
motivation for one skilled in the art to learn from Vasconcelos. Therefore, based on the 
information provided by Vasconcelos, Christel's video system would have been 
motivated to learn from Vasconcelos to identify semantic events from within video data. 
One skilled in the art at the time of the invention would have been motivated to learn 
from Vasconcelos to identify semantic events from video data and display the data in a 
timeline format. 
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Christel and Vasconcelos do not disclose displaying the identified semantic 
events in the timeline with different visual indications for two different types of semantic 
event. Ahmad discloses obtaining summary data related to video and audio data, with 
visual means for indicating a relative location for a first type of data in a video data and 
displaying the relative location for a second type of data in the video using a second 
visual indication different from the first visual indication (column 16, lines 54-65). It 
would have been obvious for one skilled in the art, at the time of the invention to learn 
from Ahmad to teach two different visual indications for two different types of 
information. Christel, Vasconcelos and Ahmad disclose systems for summarizing and 
obtaining relevant data from video and audio data, further displaying timelines for 
indicating relevant data. Ahmad further displays detailed bar information to indicate 
different visual means for different types of data. It would have been obvious for one 
skilled in the art, at the time of the invention to learn from Ahmad to implement detailed 
bar data to further provide more detailed information concerning the video and summary 
data obtained. Vasconcelos discloses an objective for providing semantic information 
so as to display data without overwhelming the user. Vasconcelos' timeline includes the 
identification of different types of semantic events through letters A, D, C and S would 
further be efficiently handled by providing two different visual indications for different 
types of semantic events. The display means of Ahmad would allow for easier 
accessing of desired data with clear displays mean for quickly identifying specific data. 
Therefore, one skilled in the art would have been motivated to learn from Ahmad to use 
different visual indications for displaying different types of semantic events. 
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Referring to claim 58, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said audio 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the audio (Figures 5 and 6). 

Referring to claim 59, Christel discloses that the indicator is modified to a portion 
of the audio that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 62, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the pluralities of segments (Figure 6). 

Referring to claim 63, Christel discloses that one of the pluralities of segments is 
the segment most temporally adjacent to the portion of the audio (Figure 6). 

Referring to claim 64, Christel discloses that one of the pluralities of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 65, Christel discloses that the plurality of segments is 
the previous temporally related segment (Figure 6). 

Referring to claim 66, Christel discloses that a user selects a portion of the audio 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio from the start thereof (page 5, column 1 , lines 5-11). 

Referring to claim 67, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 
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presents the portion of the audio within the plurality of segments (page 5, column 1 , 
lines 5-11 and Figure 6). 

Referring to claim 68, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 69, Christel discloses that a user selects a portion of audio not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
audio not included within the plurality of segments, and wherein the user selects a 
portion of the audio included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the audio within the plurality of segments 
(page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 70, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 71, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
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maintain the differentiation of the overlapping time periods (Figures and page 4, column 
2, lines 26-31). 

Referring to claim 77, Christel discloses a user selectable skip function skips a 
set of frames to a modified location of the video in at least one of a forward temporal 
direction or a reverse temporal direction, and displays the video at the modified location 
(Figure 6). 

Referring to claim 78, Christel discloses a user selectable skip function skips to a 
later temporal segment or a previous temporal segment, and displays video at the later 
temporal segment or the previous temporal segment, respectively (Figure 6). 

Referring to claim 79, Christel discloses that a user selectable scan function 
skips a set of frames to a modified location of the video in at least one of a forward 
temporal direction or a reverse temporal direction, and displays the video at the 
modified location, and thereafter automatically skips another set of frames to another 
modified location of the video in at least one of the forward temporal direction or the 
reverse temporal direction, and displays 
the video at another modified location (Figure 6). 

Referring to claim 80, Christel discloses that at least one of the forward temporal 
direction and reverse temporal direction are consistent with the different segments 
(Figure 6). 

Referring to claim 81, Christel discloses that the display of the video is at the 
start of the respective one of the different segments (Figure 6). 
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Referring to claim 82, Christel discloses that the display of the video is at a 
predetermined offset within the respective one of the different segments (Figure 6). 

Referring to claim 83, Christel discloses that the graphical user interface displays 
a respective image associated with at least a plurality of the different segments (Figure 
6). 

Referring to claim 84, Christel discloses that the respective image associated 
with the currently presented different segments is visually highlighted (Figure 6). 

Referring to claim 85, Christel discloses that during presentation of the video 
visually highlighted respective images are highlighted in a substantially regular interval 
while the sequence of the presentation of the video is at substantially irregular intervals 
(Figure 6). 

Referring to claim 86, Christel discloses presentation of the different segments 
may be modified by a plurality of different functions, and wherein the user may 
customize another function, not previously explicitly provided, by combing a plurality of 
the plurality of different functions into a single function (page 5, column 1, lines 9-11 and 
Figure 6). 

Response to Arguments 

5. Applicant's arguments filed 8/16/07 have been fully considered but they are not 
persuasive. 

Applicants argue that video content containing plays related to sports is not 
obvious in view of disclosure within Christel. Based on the nature of the disclosure 
including video data, where the contents of the video data include various types of 
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information including news segments, which often display sport segments including 
important plays. In view of the nature of the disclosure within Christel, the video 
segments including play would have been obvious. Applicant's arguments have 
disclosed that Christel does disclose summarizing news programs, where it is known 
that these news programs include a sport segment with plays associated with sporting 
events that are broadcast. Therefore, the use of content related to plays associated 
with sports is obvious in view of Christel. 

Applicant argues that none of the references disclose displaying relative location 
for a first type of semantic characterization of the play and a second type of semantic 
characterization of the play, using two different visual indications to display the first and 
second type. The combination of Christel, Vasconcelos and Ahmad disclose displaying 
a first and second type of semantic events and providing two different display visuals for 
these first and second types. Furthermore, the content displayed can be a sports play 
in view of what would be obvious to one of ordinary skill in the art over Christel. The 
combination of Christel, Vasconcelos and Ahmad discloses the claimed features and 
one of ordinary skill in the art could have combined these references along with what is 
obvious to one of ordinary skill to teach the claimed invention. The disclosure of 
Christel teaches video content where one of ordinary skill in the art would know that 
sports play data can be used as it represents video data, which can be summarized. 

Applicant argues that the combination of the three references has not explained 
how Christel would be enhanced by the combination. Christel has disclosed two 
different visual indications for identifying distinct video data that has been summarized 
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from a large set of data. Christel does not clearly disclose that the timelines disclose 
different types of semantic content but the combination of Christel, Vasconcelos and 
Ahmad does disclose temporal locations of different types of semantic content. The 
timelines of Christel does disclose placement of identified data displayed through a 
visual indication is placed relative to other identified video data within the timeline, 
therefore defining a temporal relationship. Furthermore, Christel has provided a 
motivation for combination with Vasconcelos and Ahmad for displaying the identified 
video data based on semantic content. Although, identified data is based on query, 
Christel does disclose that identifying data based on the multimedia context is relevant 
to what video data should be identified and displayed (page 4, column 2, lines 32-44). 
Christel has clearly disclosed that matching distinct video data locations based on the 
context of the video data is important. Furthermore, Applicant argues how automatic 
extraction of segments for the summary data can be compiled based on the 
combination of Christel, Vasconcelos and Ahmad. The automatic extraction of 
segments has not been disclosed in the claims of the present invention. 

Applicant argues that Christel does not disclose user interaction for selection of 
one of the plurality of segments. The graphical user interface of Christel does provides 
a means through which the user can interact with a slider to change the compression of 
the video data to more clearly see desired video data. Through this interaction/the user 
has selected a distinct video segment for viewing. Through the slider, the user can 
select segments to be more carefully viewed from the skim segments that are provided. 
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Applicants argue that Christel does not provide motivation to combine with 
Vasconcelos and Ahmad. Christel has provided motivation for combination with Ahmad 
and Vasconcelos by disclosing points where the context including information and 
events occurring within the video is taken into consideration for extraction of desired 
information. Furthermore, Christel's objective of matching video segments in a more 
efficient manner by relying on video context suggests a motivation where the context of 
the video data is relied upon for creating the summary of important data. See page 3, 
column 2, lines 12-20. Both context along with the time are relied upon to determine 
important data. Christel's disclosure clearly relies on video content to access and 
display the video summary. This reliance of video content suggests a motivation for 
learning to access and display further semantic content associated with the video. 

Applicants argue that Vasconcelos does not disclose semantic characterizations 
of a play being displayed using visual indications. Vasconcelos has clearly disclosed 
that the purpose of Vasconcelos' system is to rely on semantic characterization to 
identify data within large amounts of data based on content. The video data in 
Vasconcelos is browsed and accessed based on a semantic basis. See page 1, 
column 1, Abstract. Regardless of the level of the characterization that is conveyed in 
the timeline of Vasconcelos, the reference does teach displaying a timeline, which 
specify semantic events, including the display of two different characterizations of 
semantic events in video data. 

Ahmad discloses how two different characterizations of media data are displayed 
in two different formats for conveying to the user the data that is of importance. The 
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objective of Ahmad is to provide a convenient, efficient and easy to understand user < 
interface that would allow a user to identify specific desired data within large amounts of 
data. See column 1, line 63-column 2, line 13. 

Applicant argues that the combination of Christel, Vasconcelos and Ahmad does 
not teach semantic events. Vasconcelos' timeline discloses distinct characterizations of 
when certain semantic events are occurring in the movies of the timeline shown in 
Figure 2. The timeline displays when distinct characterizations of semantic events that 
associate the actions occurring in the scenes with a characterization of that scene to 
convey to the user the semantic events occurring in the movie. Regardless of the detail 
level of the semantic characterization that is occurring in Vasconcelos', the reference 
does disclose that semantic characterization of a movie is displayed to the user using 
visual indications. 

Conclusion 

6. Responses to this action should be submitted as per the options cited below: The 
United States Patent and Trademark Office requires most patent related 
correspondence to be: a) faxed to the Central Fax number (571-273-8300) b) hand 
carried or delivered to the Customer Service Window (located at the Randolph Building, 
401 Dulany Street, Alexandria, VA 22314), c) mailed to the mailing address set forth in 
37 CFR 1.1 (e.g., P.O. Box 1450, Alexandria, VA 22313-1450), ord) transmitted to the 
Office using the Office's Electronic Filing System. Any inquiry concerning this 
communication or earlier communications from the examiner should be directed to 
Namitha Pillai whose telephone number is (571) 272-4054. The examiner can normally 
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be reached on 8:30 AM - 5:30 PM. If attempts to reach the examiner by telephone are 
unsuccessful, the examiner's supervisor, John Cabeca can be reached on (571) 272- 
4048. 

All Internet e-mail communications will be made of record in the application file. 
PTO employees do not engage in Internet communications where there exists a 
possibility that sensitive information could be identified or exchanged unless the record 
includes a properly signed express waiver of the confidentiality requirements of 35 
U.S.C. 122. This is more clearly set forth in the Interim Internet Usage Policy published 
in the Official Gazette of the Patent and Trademark on February 25, 1997 at 1195 OG 
89. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Group receptionist whose telephone number is 
(571)272-2100. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
Namitha Pillai 
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